Programmable Logic Controllers (PLCs) are the brains of Industrial Control Systems (ICSes), and thus, are often targeted by attackers. While many intrusion detection systems (IDSes) have been adapted to monitor ICS, they cannot detect malicious network messages from a compromised PLC that conform to the network protocol. A domain expert needs to manually construct IDS rules to model a PLC's behavior. That approach is time-consuming and error-prone. Alternatively, machine learning can infer a PLC's behavior model from network traces, but that model may be inaccurate. This paper presents CoToRu - a toolchain that takes in the PLC's code to automatically generate a comprehensive set of IDS rules. CoToRu comprises (1) an analyzer that parses PLC code to build a state transition table for modeling the PLC's behavior, and (2) a generator that instantiates IDS rules for detecting deviations in PLC behavior. The generated rules can be imported into Zeek IDS to detect various attacks. We apply CoToRu to a power grid testbed and show that CoToRu generated rules provide superior performance compared to existing IDSes, including those based on statistical analysis, invariant-checking, and machine learning. Our prototype with CoToRu's generated rules provide sub-millisecond detection latency, even for complex PLC logic.

Today's network switches often use on-chip shared memory to increase the buffer efficiency and absorb bursty traffic.
Current buffer management practices usually rely on simple, generalized heuristics and have unrealistic assumptions of traffic patterns, since developing and tuning a buffer management policy that is suited for every pattern is infeasible. We show that modern machine learning techniques can be of essential help to learn efficient policies automatically.

In this paper, we propose Neural Dynamic Threshold (NDT) that uses reinforcement learning and neural networks to learn buffer management policies without any human instructions except for a high-level objective, e.g. minimizing average flow completion time (FCT). However, the high complexity and scale of the buffer management problem present enormous challenges to off-the-shelf RL solutions. To make NDT feasible, we develop three techniques: 1) a scalable neural network model leveraging the permutation symmetry of the switch ports, 2) an action encoding scheme with domain knowledge, and 3) a cumulative-event trigger mechanism to achieve efficient training and inference. Our simulation and DPDK-based switch prototype demonstrate that NDT generalizes well and outperforms hand-tuned heuristic policies even on workloads for which it was not explicitly trained.

Controlling antenna tilts in cellular networks is imperative to reach an efficient trade-off between network coverage and capacity. In this paper, we devise algorithms learning optimal tilt control policies from existing data (in the so-called passive learning setting) or from data actively generated by the algorithms (the active learning setting). We formalize the design of such algorithms as a Best Policy Identification (BPI) problem in Contextual Linear Multi-Arm Bandits (CL-MAB). An arm represents an antenna tilt update; the context captures current network conditions; the reward corresponds to an improvement of performance, mixing coverage and capacity; and the objective is to identify, with a given level of confidence, an approximately optimal policy (a function mapping the context to an arm with maximal reward). For CL-MAB in both active and passive learning settings, we derive information-theoretical lower bounds on the number of samples required by any algorithm returning an approximately optimal policy with a given level of certainty, and devise algorithms achieving these fundamental limits. We apply our algorithms to the Remote Electrical Tilt (RET) optimization problem in cellular networks, and show that they can produce optimal tilt update policy using much fewer data samples than naive or existing rule-based learning algorithms.

A promising approach for leveraging the flexibility and mitigating the complexity of future telecom systems is the use of machine learning (ML) models that can analyse the network performance, as well as taking proactive actions. A key enabler for ML models is timely access to reliable data, in terms of features, which requires pervasive measurement points throughout the network. However, excessive monitoring is associated with network overhead. Considering domain knowledge may provide clues to find a balance between overhead reduction and meeting requirements on future ML use cases by monitoring just enough features. In this work, we propose a method of unsupervised feature selection that provides a structured approach in incorporation of the domain knowledge in terms of policies. Policies are provided to the method in form of must-have features, that is the features that need to be monitored at all times. We name such family of unsupervised feature selection as policy-induced unsupervised feature selection as the policies inform selection of the latent features. We evaluate the performance of the method on two rich sets of data traces collected from a data center, and a 5G-mmWave testbed. Our empirical evaluations points at the effectiveness of the solution.

As video streaming services become more popular, users desire high perceived video quality, which has placed more stringent requirements on the quality of connection. Existing issues of cellular networks encourage users to seek alternative connections such as public Wi-Fi networks; however, expectations of both users and owners of Wi-Fi networks are not sufficiently satisfied and various concerns are yet to be addressed by a better Wi-Fi access system. Based on a go-as-you-pay scheme, we design and implement DiFi, a per-user-based system with dynamic resource allocation and pricing. DiFi offers data burst that accommodates user requirements on the burstiness of traffic, in addition to bandwidth. It better caters to the various individual requirements of users, and better utilizes the limited network resources for the owners. We leverage the blockchain-based smart contract to address realistic concerns on decentralized control, privacy and trustiness and our implementation is compatible with existing Wi-Fi infrastructures.

Mobile health (mHealth) applications, benefiting from the advance of mobile computing, have emerged rapidly in recent years and generated a large volume of mHealth data. However, these valuable data are dispersed across isolated devices or organizations, hindering discovering meaningful insights underlying the aggregated data. In this work, we present the first online data \underline{VA}luation and \underline{P}ricing mechanism, namely VAP, to incentivize users to contribute mHealth data for machine learning (ML) tasks. Under the framework of Bayesian ML, we propose a new metric based on the extent of uncertainty reduction of the model parameters to evaluate data valuation during the model training process. In proportion to the data valuation, we then determine payments as compensations for users in an online manner. We formulate this pricing problem as a contextual multi-armed bandit with the goal of profit maximization, and propose a new algorithm based on data characteristics. We also extend VAP to general ML tasks, such as deep neural network. We evaluated VAP on two real-world mHealth data sets. Evaluation results show that VAP outperforms the state-of-the-art valuation and pricing mechanisms in terms of online calculation and extracted profit.

Many efforts have been devoted to online pricing mechanism design for different settings. In this work, we consider a common but challenging setting where the buyers have private time-sensitive valuations and the seller has limited supply. The seller offers a take-it-or-leave-it posted price for each arriving buyer and aims to maximize the expected total revenue. The unknown distribution of time-sensitive valuations and limited supply significantly increase the difficulty of searching the optimal dynamic posted prices. Given B identical items to sell, when the time-dependent valuations can be estimated with a factor of α, we prove Ω(log(1/α)) lower bound with respect to the optimal fixed distribution over prices and design an algorithm achieving tight O(log(1/α)) competitive ratio. When the seller has no information about the future trends of buyers' valuations, we prove Ω(log B) lower bound and show that there is an algorithm with tight O(log B) competitive ratio by modeling the problem as adversarial bandits with knapsack optimization.
Extensive simulation studies show that our algorithm outperforms previous mechanisms in various settings.

An Internet of Things (IoT) system can include several different types of service providers, who sell IoT service, network service, and computation service to customers, either jointly or separately. The complicated coupling among these providers in terms of pricing and service decisions is an under-explored research area, the understanding of which is critical to the success of IoT networks. This paper studies the impact of the provider interaction structures on the overall IoT system with massive heterogeneous customers. Specifically, we consider three interaction structures: coordinated, vertically-uncoordinated, and horizontally-uncoordinated structures. Despite the challenging non-convex optimization problems involved in modeling and analyzing these structures, we successfully obtain the closed-form optimal pricing strategies of providers in each interaction structure. We prove that the coordinated structure is better than two uncoordinated structures for both providers and customers, as it avoids selfish price markup behaviors in uncoordinated structures. When customers' demand variance is large and utility-cost ratio is medium, vertically-uncoordinated structure is better than horizontal one for both providers and customers, due to the complementary providers' competition in horizontally-uncoordinated structure. Counter-intuitively, we identify that providers' optimal prices do not change with their costs at the critical point of customers' full participation in the vertically-uncoordinated structure.

This paper introduces a new theoretical framework for optimizing second-order behaviors of wireless networks. Unlike existing techniques for network utility maximization, which only considers first-order statistics, this framework models every random process by its mean and temporal variance. The inclusion of temporal variance makes this framework well-suited for modeling stateful fading wireless channels and emerging network performance metrics such as age-of-information (AoI). Using this framework, we sharply characterize the second-order capacity region of wireless access networks. We also propose a simple scheduling policy and prove that it can achieve every interior point in the second-order capacity region. To demonstrate the utility of this framework, we apply it for an important open problem: the optimization of AoI over Gilbert-Elliot channels. We show that this framework provides a very accurate characterization of AoI. Moreover, it leads to a tractable scheduling policy that outperforms other existing work.

With the development of Mobile Edge Computing (MEC) and Internet of Things (IoT) technology, various real-time monitoring and control applications are deployed to benefit people's daily life. The performance of these applications relies heavily on the timeliness of collected environmental information, which can be effectively quantified by the recently introduced metric named age of information (AoI). Although extensive researches have been conducted to optimize AoI under various circumstances, these works commonly require a priori information about the system dynamics that is usually unknown in realistic situations. To design a more practical scheduling algorithm, in this paper, we formulate the AoI minimization problem as a Constrained Markov Decision Process (CMDP) which can be solved by Reinforcement Learning (RL) algorithms without prior knowledge. To improve the running efficiency, we (1) introduce post-decision states (PDSs) to exploit the partial knowledge of the system's dynamics, (2) perform a batch update in every learning step, (3) decompose the system-level value function into multiple device-level value functions, and (4) propose a heuristic algorithm to find the greedy action. Numerical results demonstrate that our algorithm is highly efficient and outperforms the benchmarks under various scenarios.

An Autonomous Driving System (ADS) uses a plethora of sensors and many deep learning based tasks to aid its perception, prediction, motion planning, and vehicle control. To ensure road safety, those tasks should be synchronized and use the latest sensing data, which is challenging since 1) different sensors have different sensing periods, 2) the tasks are inter-dependent, 3) computing resource is limited. This work is the first that uses Age of Information (AoI) as the performance metric for task scheduling in an ADS. We show that minimizing AoI is equivalent to jointly minimizing the response time and maximizing the throughput. We formally formulate the AoI-centric task scheduling problem. To derive practical scheduling solutions, we extend the formulation and formulate the optimal AoI-centric periodic scheduling problem with a given cycle. A reinforcement learning-based solution is designed accordingly. With experiments simulated according to the Apollo driving system, we compare the scheduling performance of the AoI-centric task scheduling with Apollo's schedulers from the perspective of AoI, throughput, and worst case response time. The experiment results show that the maximum AoI in the proposed scheduling solution with 4 cores is lower than that in Apollo's schedulers with 8 cores.

Mobile Crowdsensing (MCS) with smart devices has become an appealing paradigm for urban sensing. With the development of 5G-and-beyond technologies, unmanned aerial vehicles (UAVs) become possible for real-time applications, including wireless coverage, search and even disaster response. In this paper, we consider to use a group of UAVs as aerial base stations (BSs) to move around and collect data from multiple MCS users, forming a UAV crowdsensing campaign (UCS). Our goal is to maximize the collected data, geographical coverage whiling minimizing the age-of-information (AoI) of all mobile users simultaneously, with efficient use of constrained energy reserve. We propose a model-based deep reinforcement learning (DRL) framework called "GCRL-min(AoI)", which mainly consists of a novel model-based Monte Carlo tree search (MCTS) structure based on state-of-the-art approach MCTS (AlphaZero). We further improve it by adding a spatial UAV-user correlation extraction mechanism by a relational graph convolutional network (RGCN), and a next state prediction module to reduce the dependance of experience data. Extensive results and trajectory visualization on three real human mobility datasets in Purdue University, KAIST and NCSU show that GCRL-min(AoI) consistently outperforms five baselines, when varying different number of UAVs and maximum coupling loss in terms of four metrics.

Recent years have witnessed tremendous growth of video streaming applications. To describe users' expectations of videos, QoE was proposed, which is critical for content providers. Current video delivery systems optimize QoE with ABR algorithms. However, ABR is usually designed for an abstract "average user" without considering that QoE varies with users. In this paper, to investigate the difference in user preferences, we conduct a user study with 90 subjects and find that the average user can not represent all users. This observation inspires us to propose Ruyi, a video streaming system that incorporates preference awareness into the QoE model and the ABR algorithm. Ruyi profiles QoE preference of users and introduces preference-aware weights over different quality metrics into the QoE model. Based on this QoE model, Ruyi's ABR is designed to directly predict the influence on metrics after taking different actions. With these predicted metrics, Ruyi chooses the bitrate that maximizes user-specific QoE once the preference is given. Consequently, Ruyi is scalable to different user preferences without re-training learning models for each user. Simulation results show that Ruyi increases QoE for all users with up to 20.3% improvement. Testbed experimental results show that Ruyi has the highest ratings from subjects.

The fast development of mobile edge computing (MEC) and service virtualization brings new opportunities to the deployment of interactive applications, e.g., VR education, stream gaming, autopilot assistance, at the network edge for better performance. Ensuring quality of experience (QoE) for such services often requires the satisfaction of multiple quality of service (QoS) factors, e.g., short delay, high throughput rate, low packet loss. Nevertheless, existing mobile edge networks often fail to meet these requirements due to the mobility of end users and the volatility of network conditions. In this paper, we propose a novel scheme that both reduces delay and adjusts data throughput rate for QoE enhancement. We design an online service placement and throughput rate adjustment (SPTA) algorithm which coordinately migrates virtual services while tuning their data throughput rates based on real-time bandwidth fluctuation. By implementing a small-scale prototype supporting stream gaming at the edge, we show the necessity and feasibility of our work. Based on experimental data, we conduct real-world trace driven simulations to further demonstrate the advantages of our scheme over existing baselines.

Mobile Live Streaming (MLS) services are now one of the most popular types of mobile apps. They involve a (often amateur) user broadcasting content to a potentially large online audience via unreliable networks (e.g., LTE). Although prior work has focused on viewer-side behavior, it is equally important to study and improve broadcaster operations. Using detailed logs obtained from a major MLS provider, we first conduct an in-depth measurement study of uploading behavior. Our key findings include large wasteful uploads, strong viewing locality, and traffic dominance of loyal viewers. Specifically, 33.3% of uploads go unwatched, and the viewership of broadcasters tends to be localized to a small set of broadcaster-specific network regions. Inspired by our findings, we propose two system innovations to streamline MLS systems: adaptive uploading and edge server pre-fetching. These optimizations leverage machine learning for reduced waste and improved QoE. Trace-driven experiments show that the adaptive uploading reduces the resources wastage by 63%, and the pre-fetching boosts the startup by 29.5%.

The rapid growth of mobile video traffic and user demand poses a more stringent requirement on efficient bandwidth allocation in mobile networks where multiple users may share a bottleneck link. This provides content providers an opportunity to optimize multiple users' experiences jointly, but users often suffer short connection durations and frequent handoffs because of their high mobility. This paper proposes an end-to-end scheme, VSiM, to support mobile video streaming applications in heterogeneous wireless networks. The key idea is allocating bottleneck bandwidth among multiple users based on their mobility profiles and Quality of Experience (QoE)-related knowledge to achieve max-min QoE fairness. Besides, the QoE of buffer-sensitive clients is further improved by the novel server push strategy based on HTTP/3 protocol without affecting the existing bandwidth allocation approach or sacrificing other clients' view quality. We evaluated VSiM experimentally in both simulations and a lab testbed on top of the HTTP/3 protocol. We find that the clients' QoE fairness of VSiM achieves more than 40% improvement compared with state-of-the-art solutions, i.e., the viewing quality of clients in VSiM can be improved from 720p to 1080p in resolution. Meanwhile, VSiM provides about 20% improvement of average QoE.